The present disclosure relates to an image processing device, an image processing method, and a program. More specifically, the present disclosure relates to an image processing device, an image processing method, and a program, each of which reduces a flicker occurring in an image, such as bright and dark stripes.
If, using an imaging element (image sensor) whose exposure timing is different based on pixel positions of the imaging element, image capturing is performed under a light source where a rapid light-dark change occurs in the brightness thereof with a lapse of time, a bright and dark striped noise is recorded in an image. In a case of capturing a moving image including successive image frames, bright and dark stripes are generated at different positions of individual image frames, and when the captured moving image is displayed on a display, horizontal stripes flowing in an up-down direction are observed.
As an example of the light source where a light-dark change occurs, a fluorescent light turned on by an alternating-current power supply is cited. A commercial power supply is 50 Hz in the eastern Japan, a commercial power supply is 60 Hz in the western Japan, and the fluorescent light repeats blinking in accordance with a change in the power-supply voltage.
If, under the illumination of such a fluorescent light, an image is captured by a camera equipped with an XY address scanning type imaging element such as, for example, a complementary metal oxides semiconductor (CMOS) imaging element, striped luminance unevenness or color unevenness is generated in an image signal.
In addition, the XY address scanning type imaging element is an element where exposure processing is sequentially performed so as to be headed from an upper row to a lower row in an image frame or from the lower row to the upper row therein. If image capturing utilizing such an imaging element is performed, bright and dark stripes corresponding to the amounts of incident light from illumination at individual exposure timings appear because the exposure timing is different in each of rows ranging from the upper row to the lower row in one image frame.
If a video (moving image) is image-captured under a lighting environment blinking with a predetermined frequency and the image-captured video is displayed on a display device, a phenomenon where a stripe pattern including a portion whose luminance is high and a portion whose luminance is low flows on a screen is confirmed. The reason is that the appearance positions of bright and dark stripes corresponding to the amounts of incident light from illumination are displaced between individual image frames configuring the moving image.
In this way, the bright and dark stripes of an image, generated based on a change in the amount of light of illumination at the time of image capturing is called a so-called “flicker”.
In addition, the “flicker” means the overall flickering of a screen, generated at the time of displaying an image on a display. The above-mentioned bright and dark stripes emerging in an image are an example of the flicker. The bright and dark stripes appearing in a captured image are generated in various patterns based on a correspondence relationship between a lighting environment at the time of image capturing and the image-capturing frame rate of a captured image.
In other words, that a fluorescent light connected to a commercial power supply (alternating-current) basically repeats blinking with a cycle twice as large as a power-supply frequency and the operating principle of the imaging element cause the generation of the flicker.
As a technique of the related art disclosing processing for avoiding or reducing such a flicker, there is, for example, the following document.
Japanese Unexamined Patent Application Publication No. 5-130551 discloses a method for utilizing the periodicity of the light and darkness of a flicker generated in a moving image and reducing the flicker by averaging and recording the pixel values of a plurality of successive frames so as to cancel out a bright and dark component.
However, in order to execute this processing, it is necessary to perform an arithmetic operation for averaging a plurality of pixel values situated in corresponding positions of a plurality of image frames, and it becomes necessary to add a signal processing circuit for this arithmetic processing. In addition, other correction processing is available that, for example, detects bright and dark stripes being generated in one image frame without applying a plurality of image frames, performs pixel value correction based on a detection result, and removes a flicker. However, in a case of performing this processing, a signal processing circuit for performing the pixel value correction becomes necessary after all.
In addition, as techniques of the related art disclosing reduction processing for flickers occurring from blinking of a light source used for displaying in a display device, blinking of a light source to which the display device itself is subjected in a viewing environment, rewriting drive processing of the display device, and so forth, there are the following documents.
There are Japanese Unexamined Patent Application Publication No. 2003-280606, Japanese Unexamined Patent Application Publication No. 2003-302947, Japanese Unexamined Patent Application Publication No. 3-138616, Japanese Unexamined Patent Application Publication No. 2009-216900, Japanese Unexamined Patent Application Publication No. 7-306397, and so forth.
Any one of processing operations described in these documents discloses reduction of a flicker caused by a lighting environment or the like at an execution timing of image display processing to be performed on a display device. None of these documents discloses processing for reducing a flicker component included in a display content itself, in other words, bright and dark stripes included in the content itself owing to a lighting environment at the time of image capturing.